a) Field of the Invention
The invention relates to a process and device for depth selection in microscope images.
b) Description of the Related Art
It is known to carry out depth measurements in microscope images as well as to suppress image portions located outside of the focal plane by means of structured illumination (AXIOMAP by ZEISS, WO 97/06509, WO 98/45745).
In U.S. Pat. No. 5,493,400 (DE 9308486U), an inclined grating projection is generated by means of wedge-shaped glass bodies with changeable orientation.
These methods are based on methods derived from interference microscopy in which periodic structures projected in the object plane are converted to interference patterns and evaluations which are made possible in this way are carried out.
For this purpose, gratings are imaged in the object planes and displaced by integral fractions of the grating constant either continuously over a certain period of time (phase shift method) or stepwise (phase step method). An image is acquired in every time interval or of every position of the grating by means of a pixel-synchronous CCD camera and the images are calculated together.
The conditions are assessed most simply when a grating with a cos2-shaped or sin2-shaped intensity variation is shifted three times by xc2xc grating constant in the direction of the grating periodicity, e.g., in the x-direction. (The method functions in principle with shifts by each integral fraction  less than xc2xd of the grating constant).
The image of the initial position is interpreted as an image with sinusoidal intensity modulation vertical to the direction of the grating bar or line (sin x), the next as cos x, and the following as xe2x88x92sin x and the last as xe2x88x92cos x.
Therefore, information is provided for every image point representing the brightfield intensity multiplied by the respective grating phase. Further relevant information can be derived from this output data.
The four images can be described as follows:
1. Ixy1=0.51obxy*(1+m*sin x)
2. Ixy2=0.51obxy*(1+m*cos x)
3. Ixy3=0.51obxy*(1xe2x88x92m*sin x)
4. Ixy4=0.51obxy*(1xe2x88x92m*cos x),
where Iobxy is the intensity (reflectivity, transmission, fluorescence) contributed by the object location x, y to the imaging, and m is the modulation factor of the grating imaging. Subtracting 3 from 1 and 4 from 2 gives:
I. Ixy=Iobxy*m*sin x
II. Ixy=Iobxy*m*cos x.
Like components vanish with this procedure.
Conversely, by adding 3 to 1 and 4 to 2, the modulation of the image content with the angle functions vanishes in both cases. Brightfield images A1 and A2 are obtained:
A1: Ixy=Iyx3+Ixy1=Iobxy,
A2: Ixy=Ixy4+Ixy2=Iobxy.
i.e., two complete brightfield images are contained in these four partial images.
The modulation m with the angle functions has impressed information on these images about the distance from the exact focal plane. The modulation factor m of the grating imaging changes depending on the utilized grating constant, objective aperture, wavelength and the distance from the focal plane.
When using an objective, a light source and a grating, the first influencing factors remain constant, i.e., the modulation factor in a given arrangement is a function of focusing.
This modulation factor multiplied by the brightfield information can be calculated from I. and II. by means of Pythagorean trigonometry by individual squaring followed by addition:   m  =                    I        obxy            *                                                  sin              2                        ⁢            x                    +                                    cos              2                        ⁢            x                                      I      obxy      
When m is taken solely as a function of location, a xe2x80x9cdistance imagexe2x80x9d of the image points from the optimum focal plane is obtained. Omitting division by the brightfield image, information is obtained which is virtually identical to that which can be obtained in confocal microscopes. The object characteristics are presented with decreasing intensity as the distance from the object plane increases, i.e., a determined layer of the object is shown.
Since the modulation factor mxe2x89xa61, the depth selectivity can be increased by raising to a higher power.
By means of selecting limits, setting threshold values for display, it is possible to represent a layer of varying thickness by itself in a synthetic image in that everything located at a distance from the focus is excluded from the display by the threshold values.
This possibility is limited by the signal/noise ratio, the step errors in the grating displacement, the nonlinearities in detection, and the number of nodes by which the modulated grating is represented by the CCD camera generally used.
In principle, the following information can be obtained from the object from the partial images described above:
1. Complete brightfield images without modulated grating structure (A1, A2);
2. Distance information of object areas from the exact focus (without direction) (m);
3. Brightfield information of object becoming darker with increasing distance from the focus (confocal images) (A*m);
4. Images in which only a thin layer around the focus is shown as a complete brightfield image (A where m greater than limit value);
5. By raising m to a higher power, a selectable smaller area around the focus.
The apparatus described in U.S. Pat. No. 4,884,881 is an arrangement belonging to the large group of spinning disk arrangements, as they are called. These arrangements require an intermediate imaging (Relaigh) system. This means that the microscope must be expanded by additional optics and accordingly rendered more expensive.
In order to achieve usable confocal effects in arrangements with a spinning disk, structures must be projected into the object plane in such a way that they have a size in that location which is comparable to the resolution limit of the arrangement. Further, the coverage of a structured mask should be no larger than 5% in order to suppress the xe2x80x9cbreaking throughxe2x80x9d of information located outside of the focal plane. This precaution leads to a very low light yield or light efficiency (xe2x89xa65%). In the this patent, wedge-shaped slits are imaged in the object plane. As a result of this construction, the confocal effect varies in radial direction (with respect to the image of the disk). In the present invention, because of the cos2-(sin2)-shaped intensity distribution of the pattern, the light efficiency is almost 50% and the achievable confocal effect is not spatially dependent in the displayed field. The device can be added to any reflected-light microscope or fluorescence microscope with an accessible field diaphragm plane in the illumination beam path without requiring additional optics.
U.S. Pat. No. 5,831,736 describes a confocal laser scanning microscope in which not all of the radiation returning from the object is detected, but only an angular area limited to the pupil plane. This area is cut out by a sector diaphragm whose direction can be changed in a stepwise manner. Based on the waveform of the detected signal depending on the orientation of the sector diaphragm, spatially dependent information about the inclination of the object structures can be obtained in addition to the depth information that can be obtained in microscopes of this type; however, because of the reduced aperture on the detection side, this information is obtained with decreased lateral and depth resolution.
In U.S. Pat. No. 5,471,308 plane plates with different inclinations are slid or rotated into the beam path of the projector successively in front of a grating pattern so that a different position of the projected pattern occurs with every one of the inclined plane plates (described in optics texts as plane plate micrometer). The apparatus described in the patent has two disadvantages:
1. Because of the large paths to be traveled and the large masses to be moved, the changing of the plates is carried out too slowly for modern applications.
2. Different optical parts are used which are subject to tolerances. In critical applications, this leads to errors which cannot be compensated in practice.
The aim of the apparatus described in U.S. Pat. No. 5,307,153 differs from that of the present invention. This apparatus is concerned with changing the grating constant of a pattern. The patent makes use of Abbe""s theory of imaging and simulates what was realized by Abbe with his diffraction apparatus for demonstrating microscope imaging theory. Changing the grating constant in depth measurement with structured illumination, as it is called, is required in order to enlarge the detectable depth range in macroscopic objects.
The arrangements described in both of the above-mentioned patents suffer from a common disadvantage. In order to detect the depth of objects, the parallax must be used, i.e., the pattern must be projected with inclined principal beam direction. It has been shown that surfaces which are inclined toward the observation direction are reproduced in a distorted manner with this method. This disadvantage can be compensated with simple objects, but not with complicated ones. Errors of this type can be compensate almost completely with projection from two directions at an inclination opposite to the observation axis.
The arrangements of U.S. Pat. No. 5,493,400 shown in FIGS. 4 and 5 are used for this purpose. The disadvantage of the arrangements shown in this reference consists in that the change in the projection direction requires an independent drive and large paths must be traveled and large masses must be moved. Therefore, the attainable speeds do not meet modern requirements.
With central illumination, the images contain no information about whether an image point lying outside of the focal plane is extrafocal or intrafocal. It is possible, however to obtain this information through oblique grating projection in an advantageous manner and a principal object of the present invention is directed to this approach.
Depending on the object height, the grating lines are shifted more or less in the direction of incidence of the illumination.
Phase information xcfx86xy by pixel can be obtained from the above equations I. and II. by division analogous to examination by interference microscopy.       ϕ    xy    =      arctan    ⁢                            sin          ⁢                      xe2x80x83                    ⁢          x                          cos          ⁢                      xe2x80x83                    ⁢          x                    .      
In so doing, the modulation factor and object intensity vanish. Therefore, it can be decided whether the respective pixel is located above or below the focal plane. It can be decided, based on the mathematical sign of p, whether the respective pixel lies above or below the focal plane. Whether a positive sign signifies above or below, and vice versa, depends on given technical factors pertaining to equipment.
Experience has shown that reconstruction algorithms for the surface profile exhibit errors or fail entirely especially in the case of highly structures objects when there are areas with low modulation or without modulation.
This last circumstance can be countered in that four images are obtained for illumination with opposite incidence. Projection-dependent distortion is compensated by adding the synthetic images.
The phase itself is not determined, but rather the mutual position of the grating lines through comparison of their position in the first quartet of images with their position in the second. In so doing, it is advantageous to determine the modulation factor in addition and to exclude from representation by mathematical means those areas whose modulation lies below a determined threshold.
If the lines of the grating in image A1 are shifted in a certain direction compared with image A2, this also signifies a direction of defocusing conditioned by factors caused by the arrangement.
In contrast to known algorithms, this procedure also enables examination of objects that are extremely structured and/or expanded in depth and, in connection with the fine adjustment available in the microscope, by acquiring images of the object from different planes, enables the three-dimensional reconstruction of such objects.
The use of oblique illumination has further advantages for depth selectivity and lateral resolution of the image. Oblique grating projection has a further advantage in case of uniformly reflecting surfaces with inclined faces.
By means of reflection, the returning information is reflected back by twice the angle of inclination of the surface elements. When using objectives with very high apertures, this also leads to a limiting of inclined areas capable of examination to xe2x89xa636xc2x0. By means of oblique projection, the usable inclination range in the methods mentioned above which do not distinguish between the intrafocal and extrafocal position of object details can be expanded by the projection angle differing by 90xc2x0.
In the AXIOMAP, a slide with a grating with cos2-shaped intensity variation could be inserted into the field diaphragm plane. The grating was displaced by a stepper motor.
A switchable carrier with two identical wedges, but with opposite orientation, was arranged between the grating and the aperture diaphragm. The inclination of illumination could be changed in this way.
A disadvantage in this method was the long switching time from one state to another. There was also no possibility of central illumination without disassembling the accessory device. These disadvantages are avoided in the new device according to present invention.
It was the object of our Invention to find an arrangement by which the phase of the grating pattern can be changed very rapidly (it must be possible to detect at least 30 individual images per second). This object was met by combining the plane plate micrometer, known per se, with a rotary scanner. This makes it possible to change the phase of the pattern with one and the same optical part in an extremely short time and with very high precision without causing further influences on the beam path. In this case, any errors which might occur can be compensated as systematic errors.
In the arrangement described in the Application, the attainable speed is increased in that the movable wedge need only be rotated by a small angle to achieve the desired change in the projection direction. Further, the change in the projection direction does not require a separate drive; rather, this object is met by suitable control of the scanner in connection with a mechanical flipflop.
It is not the object of the present invention to expand a confocal scanning microscope, but rather to transform a normal brightfield or fluorescence microscope with a normal light source into a confocal microscope with a minimum of changes.
It is an object of the present invention to find an arrangement by which the phase of the grating pattern can be changed very rapidly (it should be possible to detect at least 30 individual images per second). This object was met by combining the plane plate micrometer, known per se, with a rotary scanner. This makes it possible to change the phase of the pattern with one and the same optical part in an extremely short time and with very high precision without causing further influences on the beam path. In this case, any errors which might occur can be compensated as systematic errors.